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1 Introduction and formulation of results 

The dual billiard map is an outer counterpart of the usual billiard ball map. In 
the plane, the dual (or outer) billiard map T is defined as follows. Let M C 
be a smooth closed strictly convex curve, and let 2; be a point in its exterior. 
There are two tangent lines to M through z\ choose one of them (say, the 
right one from the view-point of z) and define T(z) to be the reflection of z 
in the point of tangency. J. Moser put forward the study of dual billiards in 
[14, 15], and by now, there exists a substantial literature on the subject: see 
[1, 4, 5, 10, 11, 13, 18, 19, 20, 21, 22, 23, 24, 25]. Exotic as they may seem, 
dual billiards are close relatives of the usual, inner ones; on the sphere, the two 
systems are isomorphic via the projective duality. 

Most of the research done on dual billiards concerned the plane case. However 
dual billiards can be defined in multi-dimensional setting as well, [19, 20, 21]. 
This definition is not widely known and is not immediately obvious, so we 
reproduce it here. One wants to replace the curve M by a hypersurface in 
Euclidean space, and then the problem is that one has too many tangent lines 
to M through a given point. This difficulty is resolved as follows. One takes 
the linear symplectic space (R^"^, lj) where a; = ^ dxi A dyi, i = 1, . . . , m, as 
the ambient space. Let M^™~^ C R^™ be a smooth hypersurface and q £ M . 
The restriction of ui to the tangent hyperplane TqM has a 1-dimensional kernel 
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^ (q) , called the characteristic line. An orientation of M induces an orientation 
of ^(g) . This direction is used as the tangent line at q to define the dual billiard 
map. 

Definition 1 Two points 2:1,2:2 £ R^™ are in the dual billiard relation with 
respect to a smooth hypersurface M if the mid-point q = (zi + Z2)/2 lies on 
M and the line {z\Z2) coincides with the characteristic line ^(g). 

In the plane (and for a convex dual billiard curve), this is the definition of the 
dual billiard map, given above. One can prove that the dual billiard relation is 
a symplectic relation. One can also prove that if M is a smooth closed strictly 
convex hypersurface then, for every point z\ outside of M, there exist exactly 
two points 22 such that z\ and Z2 are in the dual billiard relation with respect 
to M . Moreover, for one choice of the point Z2i the orientation of the ray z\Z2 
coincides with that of ^(g) , and for another the orientation is opposite (just as 
in the plane). Thus one defines the dual billiard map T, a symplectomorphism 
of the exterior M; see [19, 20, 21]. 

The dual billiard map may be considered as a discretization of the characteristic 
flow on M; the relation between the dual billiard map and the characteristic 
flow is the same as between the usual billiard map and the geodesic flow on the 
boundary of the billiard table, see discussions in [19, 20] and [21], section 4.5. 

An important problem concerning a dynamical system is to estimate the number 
of its periodic trajectories. There is a natural action of the dihedral group D„ 
on the set of n-periodic trajectories (zi, . . . , z^), Tzi = Zi+i for i = 1, . . . , n: 
one can cyclically permute the points Zi or change their cyclic order to the 
opposite. When counting periodic trajectories, we always count the number of 
D„ -orbits. 

For strictly convex smooth plane dual billiards the situation is the same as for 
the usual billiards inside a strictly convex closed smooth plane curve: for every 
period n >2 and a every rotation number 1 < r < n/2, coprime with n, there 
exist at least two distinct n-periodic dual billiard trajectories with the rotation 
number r. For the usual billiards, this is Birkhoff's theorem [3], and the result 
extends to area-preserving twist maps of an annulus, of which the plane dual 
billiard map is a particular case. 

Concerning multi-dimensional dual billiards, the only known result so far was 
the following theorem, see [19, 20, 21]: for every smooth closed strictly convex 
hypersurface in linear symplectic space and every odd prime n, the dual billiard 
map has an n-periodic trajectory. This is a rather weak estimate that can be 
considerably improved. 
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A comparison with the usual bilhards is in order. Historically the first esti- 
mate below of the number of periodic trajectories in multi-dimensional convex 
billiards is due to I. Babenko [2] where the 3-dimensional case is considered. 
This case is technically hardest, and [2] contained an error. Satisfactory lower 
bounds were recently obtained for a generic smooth closed strictly convex bil- 
liard hypersurface M'^~^ C R"^ by M. Farber and the present author in [8, 9]: 
for an odd n > 3, the number of distinct n-periodic hilliard trajectories inside 
Mrn-i j-g j^Q^ jggg than {n — l)(m — 1). Related results, in particular, on the 
number of non-closed billiard trajectories going from one point to another are 
contained in [7, 12]. 

Our goal in this paper is to estimate below the number of 3-periodic trajectories 
of a strictly convex smooth dual billiard. Period 3 is smallest possible for the 
dual billiard map, and as such, it is analogous to period 2 for the usual billiard. 
Two-periodic billiard trajectories are also called diameters or double normals: 
these arc chords of a hypersurface M™^^ C R™ that are perpendicular to M 
at both end points. If M"*^^ is smooth and strictly convex then there arc 
at least m such chords; this is one of the earliest applications of the Morse- 
Lusternik-Schnirelman theory. An extension to immersed submanifolds, not 
necessarily of codimension 1, is contained in [26], and further generalizations 
and improvements in [16, 17] and [6], where the case of generalized 3-periodic 
trajectories is considered. 

Our main result is as follows. 

Theorem 1 For every smooth closed strictly convex hypersurface M^"*~^ in 
R^"* the number of distinct 3-periodic trajectories of the dual billiard map is not 

less than 2m. This lower bound is sharp: for every m there exists a smooth 
closed strictly convex hypersurface M^"*~^ C R^"* for which this number is 
equal to 2m. 

It is an interesting problem to extend this result to greater periods, even, as 
well as odd, that is, to find a dual billiard analog of [8, 9], and to include not 
necessarily convex immersed hypersurfaces, in the spirit of [6, 12, 16, 17]. 

2 Proofs 

Our proof makes use of the critical point theory. The first step is to find a 
function whose critical points correspond to periodic trajectories. For the usual 
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billiards, this function is the perimeter length on inscribed polygons; for dual 
billiards it is the symplectic area counted with certain multiplicities. 

Consider a smooth hypcrsurfacc M^"^~^ C R^™. Let n be an odd number and 
{zi, . . . , Zn) an n-tuple of points in R^™. Set: qi = {zi + Zj-|-i)/2 where the 
index i = 1, . . . , n is understood cyclically. The following result is contained in 
[19, 20, 21]. 

Lemma 2.1 The points (zi, . . . , Zn) form an n-periodic orbit of the dual bil- 
hard relation with respect to M if and only if the points qi, ■ ■ ■ ,qn he on M 
and the n-tuple {qi, . . . , is a critical point of the function on M x . . . x M 
given by the formula: 

l<i<j<n 

Proof By Definition 1, points z^ and z^+i are in the dual billiard relation if 
and only if G M and the vector Zk+i — Zk is parallel to ^{qk)- 

It is convenient to introduce the complex structure J: 

uj{u, v) = Ju ■ V = —u ■ Jv. 

Let u{q) be the unit normal vector to M at point q £ M . Then the vector 
Ji^iq) is parallel to the characteristic line S,{q). 

Consider g = (gi, . . . , g„) G M x . . . x M. Then 

dF{q)= i-iy^'iJqidqj-Jqjdqi). (2) 

l<i<j<n 

By the Lagrange multipliers method, the point q is critical for F if and only if 

dF{q) = Y h^iQk) dqk (3) 
k 

where are Lagrange multipliers. Fix k, collect the terms in front of dqk in 
(2) and (3) and equate: 

n-2 

(-1)'= ^(-ly = -XkMqk). (4) 

j=0 

Since n is odd one can express z in terms of q: 

n-l 
j=0 

Algebraic & Geometric Topology, Volume 3 (2003) 



On three-periodic trajectories of multi-dimensional dual billiards 



997 



and hence 

n-2 

Zk+1 - Zk = 2^{-iyqk+i+j. (5) 

j=0 

Compare (5) with (4) to conclude that g is a critical point if and only if z^+i — 
has the characteristic direction at qk for all k = 1, . . . ,n. This means that the 
points Zk and Zk+i are in the dual billiard relation. □ 

Note that the function F does not depend on the choice of the origin. Note 
also that the dihedral group D„ acts on the arguments of F by permutations, 
and F is an odd function with respect to this action: F is invariant under 
cyclic permutations and changes sign if the order is reversed. This property 
distinguishes F from the perimeter length function in the usual billiard set-up 
which is invariant under For n = 3, the case under consideration here, F 
is (negative) the symplectic area of a triangle inscribed into the dual billiard 
hypersurface M: 

F{qi,Q2,q3) =(^{q2,qi) +uj{q3,q2) +uj{qi,q2)- (6) 

Assume now that M is strictly convex so that the dual billiard map T is defined. 
Most of the critical points of F{qi,...,qn) do not correspond to n-periodic 
orbits of T but rather to backtracking "fake" orbits (. . . , Zi^i, Zi, Zj+i, . . .) with 
Zi-i = Zi-\-i . This backtracking occurs when = qi . Therefore, to estimate 
the number of n-periodic orbits of T one needs to consider the critical points 
of F on the cyclic configuration space 

G(M,n) = {(gi,...,g„) G M X ... X M| T^^i for i = l,...,n} 

whose topology was studied in [8] . This critical set is acted upon by D„ , and 
each Dn orbit corresponds to a single n-periodic orbit of T. 

From now on, n = 3, the dihedral group coincides with the symmetric group 
5*3, and the function F is skew-symmetric, see (6). In particular, if qi = qj 
for some i ^ j then ^2, ^s) = 0. To prove Theorem 1 we use Morse- 

Luster nik-Schnirelman theory. One cannot immediately apply this theory to 
G{M, 3) because this space is open. A way around this difficulty is as follows. 

Choose a sufficiently small generic £ > and consider the manifold with bound- 
ary 

U = {{qi,q2,q3) e M X M X M\ 52,^3) < -e}- 

Then U C G{M, 3) . The group of cyclic permutations Z3 acts on U freely; 
let V = U/Z^. The function F descends to a function F on V , therefore 

Algebraic & Geometric Topology, Volume 3 (2003) 



998 



Serge Tabachnikov 



the number of 3-periodic trajectories of the dual bilHard map is not less than 
the number of critical points of F. The negative gradient of F has the in- 
ward direction along the boundary dV , and the usual estimates of the Morse- 
Lusternik-Schnirelman theory apply. In particular, the number of 3-periodic 
orbits of T is not less than the sum of Betti numbers of V and greater than 
the cohomological length of this space. 

The critical points of F in the set F{qi,q2, qs) > e are in one-to-one correspon- 
dence with those in U , due to skew-symmetry of the function F . However one 
may lose critical points on the zero level set, that is, 3-periodic orbits that lie 
in an isotropic affine 2-plane. In fact, no such orbits exist, as the next lemma 
shows. 

Lemma 2.2 A non-degenerate triangle with zero symplectic area, inscribed 
into M , is not critical for the symplectic area functional. 

Proof Assume that qi,q2, Qs is an inscribed triangle with zero symplectic area, 
corresponding to a 3-periodic orbit of T . Then the vector q^ — q2 is parallel to 
the characteristic direction ^{qi). Since the symplectic area vanishes, aj{q3 — 
<l2,Q2 — Qi) = 0, and hence the vector q2 — qi lies in the symplectic orthogonal 
complement to ^(gi), that is, the tangent hyperplane Tq^M . This contradicts 
strict convexity of M. □ 

We identify the homotopy equivalence class of V in two steps. Let W be the 

set of orthonormal 2-frames (61,62) in R^"^ with 0^(61,62) > 0. One has a free 
Z3 action on W by rotating a frame in the plane spanned by ei and 62 through 
27r/3 in the direction from ei to 62- 

Lemma 2.3 U is Z3 -equiVariant homotopy equivalent to W . 

Proof Given a triangle qiq2Q3 £ U , parallel translate it in such a way that its 
center of mass is at the origin. Then dilate the position vectors of the vertices 
of the triangle to unit vectors. We now have three pairwise distinct unit vectors 
zi, Z2, Z3 in R^™ that form a triangle of positive symplectic area. Let P be the 
2-planc of this triangle. 

Consider the function 

0(2:1, ^2, ^^3) = \Z1 - Z2\ + \Z2 - Zsl + \Z3 - Zi\. 

This is a smooth function on the cyclic configuration space G{S^, 3) of the unit 
circle in the plane P. The critical points of (p are 3-periodic billiard trajectories 

Algebraic & Geometric Topology, Volume 3 (2003) 



On three-periodic trajectories of multi-dimensional dual billiards 



999 



inside the circle, that is, equilateral triangles. The open manifold G{S^,3) is 
homotopically equivalent to the manifold with boundary given by the condition 

\Z1 - Z2\\Z2 - ZsWzs - Zi\>S 

where (5 > is generic and sufficiently small. The gradient of (f) has the inward 
direction along the boundary and defines a retraction of G{S^, 3) to the set of 
inscribed equilateral triangles (see, e.g., [8] for details). An equilateral triangle 
in the plane P is determined by its first vertex ei , and since the restriction 
of a; on P does not vanish, the choice of ei uniquely determines 62 so that 
a;(ei,e2) > 0. 

The above construction provides a Zs-equivariant map from U to W; this map 
gives the desired homotopy equivalence. □ 

Let A = (—1 + \/— 3) /2 be the primitive cube root of 1. Consider the standard 
action of Z3 on the sphere 5^"*"^ C C™ by multiplying every coordinate by A. 

Lemma 2.4 W is Z^-equivariant homotopy equivalent to S'^'^~^ . 

Proof We already identified R^"* with C"* by introducing the operator J. 
We want to show that W is Z3 -equivariant homotopy equivalent to the set of 
orthonormal 2-frames (61,62) with 62 = Jei. 

Consider the function V'(ci, 62) = <^(6i, 62) on the Stiefel manifold of orthonor- 
mal 2-frames. Note that V is Za-invariant. We claim that the critical points 
of consist of two critical manifolds, the complex and anticomplex frames 
62 = ±Jei. Indeed, let (61,62) be a frame and (^1,^2) its variation. Then 

61 • ui = 62 • n2 = 61 • n2 + 62 • ui = 0. (7) 

One has: 

d'tp{ui,U2) = J6i • U2 — Je2 • Ui. 

Therefore, by (7), (61, 62) is a critical point if and only if the vector (— J62, J6i) 
is a linear combination of the three vectors (ei, 0), (0, 62), (62, 61) . This is pos- 
sible only when 62 = =bJ6i. 

To complete the proof of the lemma, one chooses a generic 5 > and replaces 

W by a manifold with boundary ui{ei, 62) > 5. Then the gradient of ifj retracts 
W to the set of complex frames (ei, 62) with 62 = J6i, and this is the desired 
Z3 -equivariant homotopy equivalence. □ 
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It follows from Lemmas 2.3 and 2.4 that V is homotopy equivalent to the lens 
space S'^"^^^ /Zi^. The cohomological length of the lens space with coefficients 
in Z3 is 2m — 1. It follows from the Lusternik-Schnirelman theory that the 
function F has at least 2m distinct critical points on V . This proves the lower 
bound of Theorem 1. 

We construct a hypersurface in R^"* for which the number of 3-periodic dual 
billiard trajectories equals 2m by a small perturbation of the unit sphere 
g2m-i ^ i^2m_ describe M in terms of its support function. 

The support function of a smooth strictly convex closed hypersurface Af^™"^ 
is the function on the unit sphere h : 5"^™"^ R whose value on z G 52m- 1 
is the maximum of the linear function (f)z{x) = x ■ z on M . Let q{z) G M be 
the point at which the maximum is attained. Then 

q{z) = h{z)z + Vh{z) (8) 

where the position vector z and the gradient Vh{z) are considered as vectors 
m R2"^ . The vector z is the outward unit normal vector to M at g, and the 
characteristic direction at q is given by the vector Jz. A triple of distinct 
points zi,Z2,Z3 G g'im-i corresponds to a 3-periodic orbit of the dual billiard 
map with respect to M if and only if there exist real numbers 01,02,03 such 
that 

qi + aiJzi = qi+i - Oj+i Jzj+i, ^ = 1, 2, 3, (9) 
where qi are related to Zi as in (8). 

Lemma 2.5 Let f be a smooth Z3 -invariant function on 5*2™-! ^j^at has 
N critical Z3 -orbits. Consider the hypersurface M with the support function 
h = 1 + ef . Then, for sufficiently small e , the number of 3-periodic orbits of 
the respective dual billiard map does not exceed N . 

Proof Consider first the case when h = 1, that is, M is the unit sphere. 
Recall that A denotes the primitive cube root of 1. Then, for every z G S^"^~^ , 
the triple zi = X^z is a solution of (9) with all Oj = s/S, and every solution of 

(9) is of this form. 

Next let h = 1 + ef . Wc consider the linearizations of (8) and (9) in e. The 
number of solutions of the linearized system provides an upper bound on the 
number of genuine solutions. 

We are looking for a perturbed solution in the form 

Zi = X'iz + evi), ai = Vs + sai, i = 1,2,3, (10) 
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where z G S^^~^ and vi,V2,vs G TzS'^"^"^ (there is redundancy in the choice 
of vectors Vi, for example, one may assume that = 0). All computations are 
made modulo e^. Denote the common value of f{z),f{Xz) and f{X^z) by c. 
Likewise, one has: 

Vfiz) = X-^Vf{Xz) = X-^VfiXh), 
and we denote this tangent vector at z hy w. 
Substitute (10) to (8) to obtain: 

q{zi) = X'[z + £{vi -\- cz -\- w)], 1 = 1,2,3. 
Next, substitute to (9) and collect terms linear in e: 
(c + v^n,)- + tr + (1 + \/~^j)vi 



{c-V-'i-ai+i)z-{-w + {l-V-3)vi+i (11) 



A 

for i = 1,2,3. 

Note that A(l — \/^) = 1 + V—3, hence the coefHcients of Vi and Vi+i in (11) 
are equal. Add the three equations (11) to obtain: 

(3(1 - A)c + (1 + A) v^(ai + 02 + as)) z = 3{X- l)w. 

Since w is perpendicular to z, wc conclude that w = 0. Thus 2 is a critical 
point of the function /. Note that once such a critical point z is chosen, one 
finds from (11) that all = VSc and then solves the linear system (11) for 
vectors Vi. 

To summarize, each critical Z3 -orbit of the function / contributes a solution 
to (11), that is, corresponds, in the linear approximation, to a 3-periodic orbit 
of the dual billiard map. □ 



To prove the second statement of Theorem 1 we construct a Z3 -invariant func- 
tion on 52^1-1 ^j^jj 2m critical Z3 -orbits. Let xi, . . . ,Xm,yi, ■ ■ ■ ,ym be Eu- 
clidean coordinates in R^"* so that Xi -\- \f^yi, i = l,...,m are complex 
coordinates in C". 



Lemma 2.6 Let ai, . . . ,am be distinct reals. Then, for sufRciently small e, 
the function 

™ 2 I 2 3 Q 2 

/(a;i,...,x^,yi,...,y^) = ^a,^^+6^^^^|^ (12) 

i=l 1=1 

is Z3 -invariant on 52^.-1 ^^^j 2m critical Ti^-orhits. 
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Proof The first term in (12) is invariant under multiplication by complex 
numbers with modulus 1. The second term consists of Re ((xj + y/ —lyi)^^ , 
hence it is invariant under multiplication by A. To find critical points of / on 
the unit sphere, we use the Lagrange multipliers method: 

UiXi + e(xf - Vi) = r]Xi, aiyi - 2exiyi = rjyi, i = l,...,m, (13) 

where is a Lagrange multiplier. For each i, the system (13) has four solutions: 

^. - y. - 0- X- - V - i^S^nM (14) 

■^-i — ^ } yi — u, Xj — 2^ ) i/i — ^ v-"-^/ 

and Xi = yi = 0. The solutions (14) form a Z3 -orbit for which xj + = 
{q-aif/e^. 

For I C {1, . . . ,m} consisting of at least 2 elements, consider the function 



and let 



S = min min qrir))- 

V \I\>2 ^ ' 



Since all Oj are distinct, (5 > 0. Let e be so small that 5 > . 

We claim that for every solution of the system (13) there exists a unique index 
z G {1, . . . , m} for which (14) holds, and Xj = yj = for j i. Indeed, assume 
that (14) occurs for indices in a set I with |/| > 2. Since the domain of the 
function is the unit sphere, one has: 

1=1 iei 
contradicting the choice of e. 

Finally, there are m choices of an index i G {1, • • • ,1^1}, and for each choice 
one has (77 — ai)^ = . This gives two values of 77 and two respective solutions 
(14). Thus the function / has 2m critical Zs-orbits. □ 
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